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Introduction 


There  is  a  plethora  of  complaints  and  admonitions  concerning  Data 
Processing;  what  it  should  do  and  will  be.  Unfortunately,  most  of  these 
"discussions"  focus  on  one  DP  function  or  service  at  a  time,  leaving  each 
out  of  context.  Consequently,  trsde-offs  between  DP  functions  are  not 
explicitly  addressed  and  relative  priorities  for  resource  allocation  are 
not  established.  Amidst  this  heated  discussion,  user  and  DP  managers  look 
to  each  other  in  vain  for  policies  and  priorities. 

Consequently,  the  six  companies  studied  are  violating  the  most 
fundamental  rule  of  management  —  identify  which  activities  are  most 
important  and  allocate  resources  to  ensure  good  performance  on  those 
activities.  This  study  identifies  important  activities,  measures 
performance  on  each  activity,  and  prioritizes  management  attention  for 
improvement.  Moreover,  the  approach  used  in  this  research  is  adaptable  by 
companies  as  a  practical  foundation  for  DP  strategic  planning. 

In  most  other  functional  areas,  finance  or  manufacturing  for  example, 
we  have  a  good  understanding  of  what  constitutes  success.  Consequently  we 
can  and  do  measure  performance  in  various  ways,  assess  strengths  and 
weaknesses,  set  revised  objectives,  and  reallocate  resources  as  necessary. 
I  think  equivocation  on  strategic  planning  for  DP  is  primarily  due  to  the 
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lack  of  an  integrated  understanding  of  what  constitutes  success  for  DP. 
Defining  success  is  the  basic  prerequisite  to  performance  measurement, 
determining   priorities   for   improvement,    and  strategic  planning. 

Unfortunately,  the  definition  of  success  for  DP  is  a  moving  target. 
DP  is  being  driven  by  dramatic  growth  in  demand  for  new  application  systems 
of  more  diverse  types  .  Past  successes  have  raised  managers'  aspirations 
to  ever  higher  levels  of  information  availability  and  analysis.  Increasing 
corporate  data  resources,  continuing  improvements  in  technology,  and 
growing  DP  literacy   among   user  managers  are   all   occurring  simultaneously. 

A  continual  process  of  defining  and  measuring  DP  success  is  necessary 
to  meet  this  challenge.  We  must  identify  relevant,  current  and  emerging, 
DP  functions  and  prioritize  them.  And  we  must  continually  measure  per- 
formance in  order  to  modify  policies  and  procedures,  shift  managerial 
emphasis,   and   allocate   scarce  resources. 

In  short,  industry  needs  a  strategic  planning  process  for  DP  which  is 
strongly  linked  to  users'  needs  and  corporate  planning.  Three  major 
impediments  are  frequently  encountered  when  using  the  currently  available 
approaches: 

1.  the   strategic   planning   process  itself 

2.  lack  of   a  corporate  strategic   plan 

3.  indeterminate   implications  of  corporate  plans  for  DP 

Several  strategic  planning  methodologies  are  available;  however,  all 
are  predicated  upon  an  integrated  definition  of  success.  Every  strategic 
planning  process  is  itself  content-free.  It  is  the  responsibility  of 
management    to    define    success.        As    too    many    DP     managers    already    know. 


"User     Managers'     System    Needs,"     Robert     M.     Alloway,     CISR   Working  Paper, 
forthcoming. 
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without  an  integrated  definition  of  success,  the  outcome  of  attacking  an 
enormous  complex  planning  problem  with  a  content-free  process  is  inevitably 
inadequate. 

Unfortunately,  not  all  Chief  Information  Officers  have  access  to 
explicit  corporate  strategic  plans.  In  many  cases  this  is  simply  because 
such  plans  do  not  exist.  In  others,  the  CIO  is  not  a  regular  member  of  the 
informal  senior  management  discussions  which  evolve  implicit  strategies. 
In  some,  the  CIO  is  not  privy,  by  omission  or  commission,  to  existing 
explicit   plans  because  of  their  critical,   proprietary,   competitive  nature. 

Even  when  the  CIO  has  access  to  explicit  corporate  strategic  plans  it 
is  seldom  easy  to  divine  a  collateral  supporting  DP  strategy.  Qualitative, 
facilitative,  and  enabling  benefits  dominate  the  interpolation  and 
translation  into  DP  goals.  Opportunity  recognition  requires  intimate 
knowledge  of  current  user  activities  possessed  only  by  user  management. 
Heavy  participation  by  senior  user  managers  is  necessary  to  resolve 
priority  and  resource  allocation  trade-offs.  Total  DP  budgets  for  combined 
user  and  DP  efforts  must  be  balanced  against  goals,  personnel  availability, 
technological    trends,    and   organizational    feasibility. 

Moreover,  the  entire  DP  planning  process  should  be  quick  and 
efficient  enough  to  be  per formed/ revised  annually  without  overburdening 
scarce  management  time.  Few  companies  have  been  able  to  develop  linked 
corporate  and  DP  strategic  planning  procedures  both  effective  and  efficient 
enough  to  overcome  these   impediments. 


The  head   of  DP   will    be   referred    to    as   the    Chief    Information   Officer    (CIO) 
because   so  many  different   titles  are  used   in   practice. 
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Undaunted,  DP  and  user  management  has  achieved  tremendous  progress. 
The  typical  manager  regularly  uses  3  to  4  systems  for  access  and 

manipulation  of  information,  most  of  which  was  previously  quite  cumbersome, 

2 
unecononic,  or  simply  unavailable. 

DP  planning  does  result  in  projected  budgets,  manpower  requirements, 
and  planned  hardware  configurations.  Decisions  on  specific  projects  are 
made  in  light  of  corporate  ROI,  user  departmental  priorities,  and  3-5  year 
systems  plans.  All  of  which  _is  desirable  but  is  not  a  strategic  plan  for 
DP.  The  potential  for  improved  effectiveness  of  DP  efforts  through  a 
practical  approach  to  DP  strategic  planning  is  worth  pursuing. 

This  article  discusses  the  results  of  a  survey  of  user  and  DP 
managers  which  provides  the  specifics  for  planning  —  an  integrated 
definition  of  success,  assessment  of  strengths  and  weaknesses, 
identification  of  key  problems,  and,  based  on  the  results,  recommended 
priorities  for  action. 

The  results  from  the  six  firms  studied  are  probably  applicable  to 
your  firm  but,  even  if  you  feel  they  are  not,  this  paper  provides  a 
concrete  point  of  departure  and  a  specific  diagnostic  methodology  which 
works.  The  advantages  of  this  approach  are  its  procedural  independence 
from  corporate  strategic  planning,  strong  linkage  and  translation  of  user 
needs  to  DP  planning,  and  minimal  time/effort  required.  The  key 
disadvantage  is  its  deceptive  simplicity  and  potential  for  misuse  — 
yielding  misleading  results  for  those  who  lack  experience  and  expertise  in 
quantifying  subjective  managerial  opinions  using  questionnaire  measurement 
techniques. 
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Reseerch  Method 

Identification  of  criteria  on  which  to  measure  success  began  with 
extensive  discussions  and  "blue-skying".  The  resulting  list  was  imprac- 
tically  long  and  dominated  by  technical/operational  criteria.  Anthony's 
partitioning  of  strategic,  managerial  and  operational  activities  was  used 
to  balance  and  shorten  the  list  to  26  criteria.  I  make  no  claim  that  this 
list  is  exhaustive  or  optimal,  but  it  _is  a  good  start.  See  Exhibit  1  for  a 
list  of  the  26  criteria. 

Next,  I  needed  importance  and  performance  ratings  for  each  criteria 

4 
from  many  managers.  As  part  of  a  larger  research  project  ,  114  DP  and  user 

managers  in  6  diverse  manufacturing  firms  completed  an  extensive 

questionnaire.   Respondents  were  selected  to  be  a  stratified  sample  of 

senior,  middle  and  junior  managers  from  the  DP,  Finance  and  Manufacturing 

departments.  See  Exhibits  2  and  3  for  firm  and  respondent  profiles. 

Every  manager  rated  each  of  the  26  criteria  on  two  separate  questions 

one  for  importance  and  one  for  performance.   Relative  importance  was 

used  to  weight  performance  when  calculating  success  for  DP  and  when 

computing  success  for  each  of  the  26  criteria.   Because  every  manager  is 

known  by  name  and  organizational  role,  comparisons  between  managers  and 

departments  were  easily  made  on  individual  criteria,  groups  of  criteria, 

and  overall  success. 


Planning  and  Control  Systems,  Robert  N.  Anthony,  Harvard  Business  School, 
1965 

"User  Needs  Survey:   Preliminary  Results,"   Robert  M.  Alloway,  et  ^, 
Sloan  School  of  Management  Working  Paper,  1096-79,  December  1979. 
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Is  there  a  DP  success  problem? 

Overall  DP  success  is  mediocre.  Measured  on  the  seven  point  scale  in 
Exhibit  ii,  average  success  is  only  4.1  for  the  6  firms.  It  is  naive  to 
expect  DP  success  ratings  of  7  (excellent) ,  but  5  (good)  should  be 
realistic;  whereas  4  is  teetering,  most  firms  have  a  major  DP  shake-up 
before  3  (inadequate)  is  reached. 

Yes,  there  is  a  real  problem  in  DP  success.  These  results  are  not  a 
surprise  but  certainly  a  cause  for  concern. 

DP  managers,  naturally,  rated  their  departments  higher,  4.4,  than 
user  managers,  4.0,  but  nonetheless  they  basically  agree.  Things  are  not 
"good",  5.0,  and  definitely  not  "excellent",  7.0.  There  is  considerable 
incentive,  across  the  board,  to  address  the  definition  of  DP  success  and 
its  improvement. 

Firm  differences  in  overall  success  are  large,  ranging  from  3.5  to 
4.8.  More  importantly,  firm  differences  in  users'  ratings  of  DP  success 
are  even  larger,  ranging  from  3.2  to  4.8.  Talking  to  user  managers  in 
these  two  firms  is  the  difference  between  night  and  day.  Firm  5  at  4.8 
still  has  ample  room  for  improvement  but  improvements  are  possible, 
supported,  and  underway. 

On  the  other  hand.  Firm  3  is  in  real  trouble,  not  only  because  of  low 
user  success  ratings,  but  also  because  of  the  large  difference  between  DP 
(4.0)  and  user  (3.2)  success  ratings.  A  large  difference  between  DP  and 
user  ratings  makes  problem  recognition  and  cooperation  for  improvement 


Firm  3,  for  example,  underwent  a  fundamental  transfer  of  responsibility 
away  from  DP  almost  concurrent  with  our  data  gathering. 
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significantly  more  difficult.  The  management  of  firm  3  decided  it  was 
necessary  to  completely  restructure  DP  in  order  to  overcome  this  barrier  to 
improvement. 

Firms  1  and  6  also  have  large  differences  between  user  and  DP  success 
ratings,  5.1  to  A. 4  and  4.6  to  3.6  respectively.  Without  common  problem 
recognition  and  cooperation,  DP  and  users  become  entrenched  in  attack- 
defense  efforts  thereby  stagnating  improvement  until  success  becomes  so  low 
a  radical  restructuring  of  DP  is  finally  forced. 

Common  knowledge  recognizes  a  widespread  problem  with  DP  success, 
user  dissatisfaction,  and  important  differences  in  DP  success  among  firms. 
This  research  confirms  this  DP  problem  with  empirically  measured  importance 
and  performance  on  26  criteria  integrated  into  an  overall  definition  of  DP 
success.   This  alone  is  a  significant  contribution. 

The  real  contribution,  however,  derives  from  what  can  be  learned  by 
dis-aggregating  overall  success.  For  the  first  time,  within  an  integrated 
definition  of  DP  success  we  can  dig  deeply  into  patterns  of  strengths  and 
weaknesses,  compare  firms,  deal  with  trade-offs  between  criteria,  and 
search  for  the  causes  of  low  success. 
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What  are  DP's  strengths  and  weaknesses? 

In  order  to  diagnose  causes  of  low  success  we  must  begin  with 
identification  of  major  areas  of  strength  and  weakness.  This  was 
accomplished  by  focusing  on  users'  average  ratings  of  DP  success,  and 
grouping  the  26  criteria  to  differentiate  between  areas  of  relative 
strength  and  weakness. 

Cynical  opinions  notwithstanding,  user  managers  are  quite  capable  of 
differentiating  among  various  DP  functions.  Exhibit  1  lists  average  users' 
success  ratings  for  each  criteria.  The  range  of  success  across  the 
criteria  is  even  greater  than  the  range  in  firm  success.  Good  success 
(5.0)  on  criterion  H  (technical  competence  of  the  DP  staff)  is 
significantly  different  from  inpdequate  success  (2.7)  on  criterion  D 
(training  programs  for  users  in  DP  capabilities).  Users  can  and  do 
recognize  criteria  where  DP  is  successful  and  distinguish  those  from  other 
criteria  where  it  is  not.  All  DP  activities  are  not  "tarred  with  the  same 
brush"  by  the  user  managers  studied. 

The  26  criteria  were  clustered  into  the  8  groups  in  Exhibit  1  based 
on  common  patterns  in  user  managers'  performance  ratings.  Users 
conceptually  group  like  DP  activities  together  and  distinguish  between 
management/process  problems,  user  relations,  and  ongoing  daily  activities. 
This  makes  a  significant  point.  Users  understand  DP  activities  well  enough 
to  differentiate  among  them.  User  managers'  opinions  are  not  naive  or 
unrelated  to  the  realities  of  DP.  Users  opinions  are  a  valid  source  of 
relevant  importance  and  performance  information  for  diagnosis  and  planning. 


Obviously,  the  26  criteria  have  been  rearranged  from  the  original 
questionnaire.  A  product-moment  principal  factor  analysis  was  used  for 
grouping. 
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The  managerial,  effectiveness-oriented  groups  of  criteria  (e.g. 
Development  and  User  Relations)  are  less  successful  from  the  users'  point 
of  view  than  the  traditional,  efficiency-oriented  groups  (eg.  Technical  and 
Operations).  According  to  these  user  managers  the  essence  of  effectiveness 
improvement  in  DP  is  the  development  of  more  managerially-oriented 
application  systems.  This  necessitates  better  user  communications, 
training,  responsiveness,  and  attitudes  as  well  as  supporting  proposal 
preparation,  improving  the  systems  development  process  itself,  and 
restructuring   the   systems  development  backlog. 

Do  strengths  and  weaknesses  vary  by  Firm? 

I  expected  the  patterns  of  strengths  and  weaknesses  to  be 
significantly  different  between  high  and  low  success  firms.  The  hypothesis 
was  that  high  success  DP  departments  would  do  well  on  both  efficiency  and 
effectiveness  criteria,  whereas,  the  pattern  for  low  success  DP  departments 
would  be  good  efficiency  but  poor  effectiveness.  This,  however,  is  not 
what  the  data  says. 

Firms  3  and  5  were  selected  to  contrast  the  differences  between  high 
and  low  success  firms  using  the  eight  groups  of  criteria  identified  in 
Exhibit  1.  Refer  to  Exhibit  5  for  user  success  ratings  by  group  for  these 
two  firms.  Firms  3  and  5,  with  overall  success  of  3.2  and  4.8  respec- 
tively, have  the  same  basic  pattern  of  strengths  and  weaknesses  —  only  the 
levels  differ.  Both  firms  have  the  same  relative  strengths  in  efficiency 
and  weaknesses  in  effectiveness. 

This  pattern  holds  for  all  six  firms  studied.  From  the  researcher's 
point-of-view  this   common   pattern  of   strengths   and   weaknesses   turns  out   to 
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be fortuitous.  One  set  of  recommendations  for  improvement  will  apply  to 
all  six  firms  and  it  lends  credibility  to  generalizations  to  other  firms. 

Average  user  ratings  of  DP  success  by  criteria  show  a  pattern  of 
relative  strength  in  efficiency  and  weakness  in  effectiveness.  All  six  DP 
departments  studied  have  this  same  pattern  of  relative  strengths  and 
weaknesses.  This  is  a  terribly  helpful  diagnosis,  a  description  of 
relative  success  patterns  by  criterion  and  firm.  It  is  not,  however,  the 
identification  of  the  causes  for  low  success.  For  causes  we  must  dig 
deeper,  further  dis-aggregating  success  in  a  search  for  causes  of  low 
success. 

The  commonly  acknowledged  and  often  discussed  communication  problems 
between  users  and  DP  suggest  three  possibilities  for  search:  disagreements 
between  users  and  DP  on  relative  success,  importance,  and/ or  performance  on 
the  criteria  for  success. 
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Do  users  and  DP  agree  on  success  by  criteria? 

Surprisingly,  DP  and  users  agree  on  the  success  of  each  of  the  26 
criteria.  Exhibit  6  plots  DP  versus  user  success  ratings  on  each  criteria. 
If  either  users  or  DP  disagreed  on  any  criteria,  it  would  be  plotted  far 
from  the  diagonal.  Notice  how  the  user  and  DP  success  ratings  for  all  of 
the  26  criteria  are  tightly  grouped  along  the  diagonal.  Clearly, 
disagreements  over  success  are  not  the  cause  we  are  searching  for; 
however,  this  exhibit  is  packed  with  insightful  information  —  both 
alarming  and  hopeful. 

Alarming,  because  the  top  of  the  "ladder  to  success"  is  empty.  DP 
and  users  agree  —  nothing  even  approaches  "excellent",  and  only  technical 
competence  (H)  barely  makes  the  "good"  rung  on  the  ladder.  The  bulk  of  the 
criteria  fall  in  the  caution  zone  between  the  scale's  midpoint  and  "good"; 
they  could  go  either  way.  Seven  of  the  criteria  fall  below  the  midpoint  in 
the  clear  danger  zone  and  one,  D,  is  universally  recognized  as  below 
"inadequate" . 

Exhibit  6  is  also  hopeful.  The  necessary  prerequisite  for 
improvement,  co- joint  clarity  of  problem  identification  between  users  and 
DP,  is  fulfilled.  This  excellent  agreement  between  DP  and  users  cuts 
through  a  lot  of  the  heated  debate  and  allows  interested  parties  to 
progress  to  the  next  level  of  discussion  —  problem  causes  and  how  to  fix 
them. 

However,  all  DP  managers  are  not  equally  convinced  there  is  a  severe 
problem  nor  do  all  DP  departments  and  users  have  such  excellent  agreement 
on  success.  Exhibit  4  shows  some  material  differences  between  DP  and  user 
success  ratings,  firm  6  for  example  at  4.6  and  3.6  respectively.  Moreover, 
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the  plots  of  DP  versus  user  success  ratings  for  each  firm  (not  shown  for 
lack  of  space)  indicate  that  some  convincing  is  still  necessary  especially 
in  low  success  firms.  The  higher  the  overall  success  rating,  the  better 
the  agreement  between  DP  and  users  on  the  success  of  each  criteria.  The 
less  successful  firms  face  a  preliminary  task,  refining  co- joint  agreement 
on  problem  identification.  The  gap  between  high  and  low  success  firms  will 
continue  to  widen  unless  low  success  firms  cease  their  entrenched  attack- 
defense  efforts  and  redirect  their  efforts  to  diagnosis  and  cooperative 
improvement . 
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Is  it  importance  or  performance? 

Ratings  for  importance  of  each  criteria  for  DP  versus  users  is 
plotted  in  Exhibit  7.  Again  all  26  criteria  are  tightly  grouped  along  the 
diagonal.  This  dispels  the  possibility  that  low  success  results  from  user- 
DP  disagreements  over  importance.  There  is  excellent  agreement  on  the 
relative  importance  of  all  26  criteria!  This  does  not,  of  course,  imply 
that  these  importance  ratings  are  "right"  or  do  not  evolve.  It  does  imply 
that  user  managers'  opinions  of  the  relative  importance  of  the  macro  level 
DP  criteria  studied  are  just  as  "right"  as  DP's. 

The  plot  of  DP  versus  user  ratings  of  performance  on  each  criteria 
(not  shown  for  lack  of  space)  reveals  similarly  excellent  agreement.  With 
each  succeeding  agreement  between  DP  and  users  on  success,  importance,  and 
performance,  the  credibility  of  user  opinions  has  been  enhanced.  The 
quality  of  user  understanding  of  DP  managerial  issues,  which  has  been 
revealed  in  Exhibits  6  and  7,  is  impressive  and  potentially  beneficial  to 
both  DP  and  the  whole  firm.  User  management  is  capable  and  clearly  should 
participate  in  the  managerial  process  of  diagnosis  and  planning  for  the  26 
criteria  studied. 

So  far,  we  have  identified  26  criteria,  measured  their  importance  and 
performance,  developed  a  simple  but  reliable  integrated  definition  of  DP 
success,  revealed  the  efficiency/ effectiveness  pattern  of  strengths  and 
weaknesses  by  criteria  for  all  six  firms,  and  searched  for  the  cause  of  low 
success. 

Three  possible  causes  of  low  success  have  been  checked  and  rejected. 
DP  and  users  agree  on  success,  importance,  and  performance  for  all  26 
criteria.  The  quality  and  consistency  of  this  agreement  is  surprising 
given  all  the  "discussions"  about  unknowledgeable  users,  unresponsive  DP, 
and  DP -user  communication  problems.  The  question  remains;  what  is  the 
caiisfi  of  Inw  RiinnRSR? 
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Found:   the  cause  of  low  success 

The  cause  of  low  success  is  fundamental  and  straightforward  —  user 
managers  see  absolutely  no  relationship  between  importance  and  performance! 
Exhibit  8  plots  users'  ratings  of  importance  and  performance  for  each 
criteria.  These  criteria  should  all  lie  on  the  diagonal.  Instead,  there 
is  no  managerially  justifiable  pattern,  or  any  pattern  for  that  matter;  it 
resembles  only  a  random  shotgun  pattern. 

The  most  basic  managerial  rule,  allocate  resources  to  perform  well  on 
the  most  important  tasks,  is  clearly  being  violated.  Equally  important 
criteria,  D  and  B,  have  performances  ranging  from  good  to  inadequate. 
Equal  performance  results  are  achieved  on  criteria  Z  and  W  with  importances 
ranging  from  indifferent  to  high. 

Exhibit  9  plots  DPs'  ratings  of  importance  versus  performance  for  all 
26  criteria.  This  preempts  the  argument  that  the  users'  perceptions 
displayed  in  Exhibit  8  are,  to  put  it  politely,  inaccurate.  Again,  these 
criteria  should  all  lie  on  the  diagonal.  DP  managers  also  fail  to  detect 
any  managerially  justifiable  relationship  between  importance  and 
performance.  The  imbalance  between  importance  and  performance  is 
definitely  the  fundamental  cause  of  low  success. 

Consider  for  example,  the  development  of  monitor,  exception,  inquiry 
and  analysis  type  systems  (criteria  3,    K,  L  and  M) .   The  more  traditional/ 


For  a  complete  discussion  of  system  types,  their  growth  rates,  managerial 
appropriateness,  and  levels  of  demand  see  "User  Managers'  Systems  Needs", 
Robert  M.  Alloway,  CI5R  Working  Paper,  forthcoming. 
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efficiency-oriented  the  system  (3  and  K),  the  lower  the  importance  but  the 
higher  the  performance.  Conversely,  the  more  managerial/effectiveness- 
oriented  the  system  (L  and  M) ,  the  lower  the  performance  but  the  higher  the 
importance.  These  four  points  nearly  form  a  straight  line  sloping  downward 
to  the  right.  The  greater  the  importance  the  lower  the  performance! 
Exactly  the  opposite  of  the  relationship  we  should  see. 

Obviously  it  is  time  to  shift  management's  attention  and  reallocate 
scarce  resources  to  close  the  gap  between  importance  and  performance.  But, 
where  should  we  begin? 

What  hurts  the  most? 

DP  and  user  managers  in  these  six  firms  could  significantly  improve 
success  if  they  conducted  an  indepth  diagnosis  of  their  importance- 
performance  gaps  and  prioritized  their  reallocation  of  resources  to  improve 
performance  on  highly  important  criteria.  This  can  be  done  with  the 
approach  to  defining  DP  success  used  in  this  research. 

Exhibit  10  returns  to  firm  5  for  an  example  indepth  diagnostic  of  the 
importance-performance  gap  at  the  firm  level.  The  location  of  the  axes  at 
5  and  5  (good  and  important)  is  somevliat  arbitrary,  but  I  have  tried  to 
choose  achievable  standards.  The  upper-right  corner  is  the  relative 
"success"  quadrant.  This  implies  no  change  is  necessary  in  importance- 
performance  for  these  criteria.  The  lower-left  quadrant  is  also  "OK"  in 
the  sense  that  lower  performance  on  these  comparatively  unimportant 
criteria  is  acceptable. 

My  recommendation  for  these  two  quadrants,  leave  them  be,  might 
appear    much    too    benign    until    one    contemplates    the    level    of    managerial 
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attention  and  effort  required  to   improve  the  lower-right  quadrant. 

The  lower-right  quadrant  is  the  real  "killer".  These  criteria  have 
high  importance  but  low  performance.  These  are  the  criteria  which  ruin  a 
DP  department's  reputation,  drive  users  up  the  wall,  seriously  impair  DP's 
ability  to  deliver,  and  prevent  user  managers  from  receiving  their  relevant 
information. 

The  upper-left  quadrant  should  receive  declining  management 
attention.  Any  increased  efforts  to  improve  performance  here  are  a 
relative  "waste"  of  resources.  Clearly  DP  management  should  not  steal 
resources  from  the  "success"  quadrant  for  use  in  the  "killer"  quadrant, 
rather,   DP  should     reallocate   from  "waste"   and  "OK"  to  "killer." 

DP  in  firm  5  has  a  good  operations  group  (X,W,B,C  and  Z)  and 
traditional  technical  competence  (J,F  and  H).  In  fact,  they  are  over- 
perfonning,  in  the  "waste"  quadrant,  on  one  criteria  (3  and  Z)  in  each 
group.  Given  scarce  DP  resources,  the  leverage  for  improved  overall 
success  definitely  lies  elsewhere,  in  the  "killers",  in  managerially 
oriented  new  systems  development  (L  and  M)  to  achieve  responsiveness  to 
user's  needs  (0)  and  improve  users  attitudes  (G).  This  necessitates 
training  programs  for  users  (D)  and  support  for  users  in  proposal 
preparation  (U)  for  needs  recognition  and  project  practicability. 
Moreover,  senior  user  managers  must  be  more  involved  (N),  the  new  systems 
development  process  itself  needs  improvement  (S),  and  systems  analysts  need 
more  knowledge  of  users'  operations  (T)  for  prioritization  and  successful 
implementation. 

In  short,  a  complete  reassessment  of  the  systems  development 
activities  (policies,  portfolio  balance  by  system  type,  systems  development 
procedures,  user  training  programs,  and  systems  analysts'  skills)  is  needed 
to  make  them  more  user  management  oriented. 
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Each  firm  is,  of  course,  unique  with  its  own  special  needs  implying 
importances  and,  equally  critical,  its  own  profile  of  performance.  Firm  3 
(see  Exhibit  11)  both  differs  from  and  is  similar  to  Firm  5.  The  most 
noticeable  difference  is  lower  success  on  nearly  all  criteria  (3.2  average 
success  vs  4.8  for  firm  5)  reflected  in  a  virtually  empty  "success" 
quadrant  and  an  overwhelming  "killer"  quadrant.  The  magnitude  of  change 
required  is  simply  impractical  if  all  "killer"  criteria  are  addressed 
simultaneously. 

As  mentioned  previously,  the  location  of  the  axes  (solid  lines)  at  5 
and  5  is  arbitrary  —  so  we  can  easily  customize  for  firm  3's  average 
performance  (dashed  line).  This  enables  firm  3  to  focus  on  a  smaller,  more 
practical  subset  of  improvement  efforts. 

When  firm  3  has  improved  its  current  "killers",  they  will  be  replaced 
by  others.  Their  firm  relative  performance  standards  (dashed  line)  will 
move.  Defining  success,  assessing  importance-performance,  and  mounting 
efforts  to   improve  is  a  basic,   recurring  managerial   task. 
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Priorities  for  Action 

Given  the  excellent  agreement  between  DP  and  users  we  have  seen  and 
the  consistency  of  the  patterns  across  all  six  firms,  generalizations  of 
action  priorities  for  these  six  firms  are  reasonable. 

Improvements  in  DP  should  begin  with  those  criteria  having  the 
largest  differences  between  importance  and  performance.  Moreover,  it  will 
be  significantly  easier  to  achieve  improvements  on  a  particular  criteria  if 
both  DP  and  users  agree  on  the  relative  size  of  its  importance-performance 
difference. 

Exhibit  12  depicts  these  importance-performance  differences  between 
DP  and  user  managers  in  all  six  firms.  The  greater  the  difference,  the 
greater  the  priority  for  action.  Fortuitously,  DP  and  user  managers  have 
excellent  agreement  on  these  differences  for  all  26  criteria. 

Beginning  with  the  upper-right  corner,  the  top  three  priority 
clusters  can  be  identified.  Ironically,  the  top  priority  improvement  for 
DP  is  to  develop  a  general  education  program  in  DP  for  user  managers  (D). 
It  would  seem  logical  for  this  program  to  focus  heavily  on  the  criteria  in 
the  other  two  top  priority  clusters. 

Three  criteria  in  the  second  priority  cluster  are  more  like  outcomes 
than  inputs  —  perceived  responsiveness  (0),  communication  (A)  and  user 
attitudes  (G).  Improvements  in  the  other  top  priority  criteria  should 
favorably  affect  these  criteria.  They  can  also  be  directly  addressed  by 
explicitly  allocating  more  DP  management  time  to  communicating  with  user 
managers.  The  fourth  criterion  in  this  cluster  (M)  is  the  development  of 
more  analysis  systems.  This  type  of  application  includes  planning/ what-if. 
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statistics/ forecasting,      simulation/optimization,       and      Decision      Support 
Systems. 

The  third  priority  cluster  includes  the  other  managerial  system  type, 
flexible  inquiry  systems  (L).  These  systems  allow  user  managers  access, 
either  personally  or  via  an  intermediary,  to  ad-hoc  and  flexible  reports  on 
an     as-needed     basis.  The     potentials     of     currently     available    database 

management  systems  and  user  query  languages  should  be  aggressively  pursued. 
This  has  a  synergystic  effect  with  cleaning-up  current  report 
contents  (Y).  There  should  be  minimal  problems  with  outdated  report 
contents  from  flexible  inquiry  systems;  as  users'  needs  change,  users 
modify  their  report  contents  accordingly.  Moreover,  the  maintenance 
workload  on  output  reports  should  decline  as  traditional  monitor  (J)  and 
exception  (K)  systems  are  converted  to  flexible  inquiry  (L).  Cleaning  up 
report  contents  on  unconverted  applications  is  a  DP  "budget-eater"  but 
according  to  these  results  should  not  be  downplayed.  Rather,  management 
attention   to  the  maintenance  process  should  be  invested   to   improve  it. 

Improving  the  systems  development  process  itself  (S)  is  an  enabling 
requirement  for  all  applications  and  especially  for  creating  more  inquiry 
(L )  and  analysis  (M)  systems.  To  date,  considerable  time  and  money  has 
already  been  expended  on  improving  the  systems  development  process  with 
modest  and  sporadic  results.  A  fundamental  flaw  in  these  efforts  has  been 
to  design  the  one   project   procedure   appropriate   for   all   systems.      This  has 


see  Decision  Support  Systems,  Keen  and  Scott  Morton,  Addison-Wesley,  1978 
and  "Decision  Support  Sytems  and  Information  Flows  in  the  1980 's",  Robert 
M.   Alloway,   Teleinformatics  79,   North-Holland  Publishing,    1979. 


-20- 


resulted  in  project  procedures  appropriate  for  monitor  and  exception 
systems  (90%  of  the  installed  base)  and  inappropriate  for  inquiry  and 
analysis  systems  (40?o  of  new  systems  demand).  The  key  here  is  to 
recognize  the  fundamental  differences  in  system  types  and  develop  project 
procedures  appropriate  for  each  —  maintenance,  transaction  processors 
(monitor  and  exception),  inquiry,  and  analysis. 

Note  the  cross-functional  skill  development  implied  by  user  education 
in  DP  (D)  and  DP  education  in  user  functions  (T).  For  both  sides,  this  is 
supplemental  to  skill  development  programs  in  their  own  function. 
Identification  and  prioritization  of  skills,  effective  communication  of 
these  priorities,  and  corresponding  modification  of  personnel 
evaluation/reward  systems  is  necessary  for  success  in  these  efforts. 

Defining  success  for  DP  is  an  ongoing  process,  necessitating 
re-assessments  of  the  relative  importance  of  alternative  criteria  to  user 
management.  This  requires  the  involvement  of  senior  user  managers  in  DP 
policy  formulation  and  evaluation  (N).  Many  managers  have  mixed  feelings 
about  the  effectiveness  of  DP  steering  committees.  Perhaps,  with  some  of 
the  meaty  issues  raised  by  defining  success  for  DP,  steering  committees 
would  have  more  impact  than  serving  as  project  progress  reviewers. 

These  ten  top  priority  criteria  are  mutually  complementary  except 
that  they  compete  with  each  other  for  management  attention  and  resources. 
They  can  be  categorized  as  in  Exhibit  13  and  perhaps  addressed  by  different 
sub-groups  within  DP. 


"user  Managers'  Systems  Needs",  Robert  M.  Alloway,  CI5R  Working  Paper, 
forthcoming. 

"Temporary  Management  Systems",  Robert  M.  Alloway,  CISR  1978. 
"Grow  Your  Own:   Skill  Planning,  Development,  and  Reward  for  Systems 
Analysts",  Robert  M.  Alloway,  Datamation,  April  1980. 
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Exhibit  13 
Categorization  of  Improvement  Efforts  by  Type 


necessary 
policy 
changes 


education 

programs 

D,  T 


increased 

directed 

action 


series  of 

projects 

M,  L,  Y 


collateral 
managerial 
attention 


"outcome" 
like 


0,  A,  G 


There  is  much  more  information  contained  in  these  exhibits  than  space 
permits  discussing  in  detail.  The  reader  is  invited  to  select  his/her 
favorite  criteria  from  Exhibit  1  and  trace  them  through  the  other  exhibits. 
The  fundamental  conclusions  from  these  exhibits,  however,  have  been 
discussed. 

Summary  and  Recommendations 


The  recommendations  presented  in  this  paper  are  for  the  sample  of  six 
firms  studied.  This  is  a  sound  piece  of  research  with  clear,  emphatic 
results,  however,  generalizations  from  this  or  any  research  should  be 
approached  with  caution.  Only  six  firms  were  studied  —  all  in  the 
industrial  sector.  Only  a  stratified  sample  of  managers  in  finance, 
manufacturing  and  DP  were  included.  Respondents  were  reguired  to  guantify 
their  subjective  opinions  in  a  guestionnaire.  The  26  criteria  used  to 
define  and  measure  success  are  not  exhaustive. 
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Strategic  planning  for  DP  must  be  predicated  upon  some  definition  of 
overall,  integrated  success.  Setting  objectives,  determining  priorities, 
and  re-allocating  resources  is  fundamentally  based  upon  a  diagnosis  of 
differences  between   actual   and  desired. 

In  this  research,  performance  and  importance  represent  actual  and 
desired  for  each  criterion.  Combined  into  an  index  of  success  the  severity 
of  the  problem  can  be  gauged  by  firm,  department,  hierarchical  level,  or 
criterion.  Disaggregated,  diagnoses  of  cause  can  be  made  and  priorities 
for  improvement  assigned.  The  flexibility  to  perform  resource  trade-off 
analysis  at  the  aggregate  level  or  causal  diagnosis  at  the  dis-aggregated 
level  is  inherent  in  this  integrated  approach  to  defining  and  measuring 
success. 

The  results  of  this  study  are  dramatic  and  clear,  but  require 
material  shifts  in  managerial  attention  and  resource  allocation. 
Considerable  cooperation  from  the  right  people  is  necessary  to  address 
performance  improvements  on  the  top  ten  priority  criteria  identified.  If 
you  or  "necessary  others"  are  not  convinced  of  the  applicability  of  these 
results  to  your  firm,  then  you  must  customize  and  repeat  this  exercise  — 
define  success,  measure  it,  identify  problem  causes,  and  prioritize 
necessary  improvements.  The  process  chosen  to  accomplish  these  steps  in 
the  planning  process  is  very  important.  It  should  yield  measureable 
specifics,  be  easily  understood,  and  practical  enough  for  annual 
measurements.  It  is  quite  practical  to  use  a  customized  version  of  the 
approach  used  in  this  research  for  diagnosis,  DP  strategic  planning,  and 
prioritization  of  improvement  efforts. 

The  next  steps  in  the  planning  process  vary  somevyhat  based  upon  the 
results  of   the   preceeding   steps.      For   these   six   companies,    for   example,   it 
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is  not  necessary  to  convince  DP  there  is  a  problem,  to  develop  agreement 
between  DP  and  users  on  criteria  with  low  performance,  or  to  reconcile 
differences  in  importance  ratings  of  various  criteria  between  DP  and 
users* . 

Achievable  objectives  must  be  determined  for  each  criteria  and  the 
overall  improvement  effort.  This  is  an  iterative  process  of  assessing  the 
levels  of  improvement  possible  given  various  levels  of  available  resources. 
It  is  important  to  also  consider  the  interrelationships  between  criteria 
for  possible  precedence  relationships  and  workload  impact  on  the  personnel 
involved  as  in  Exhibit  13. 

Strategic  planning  for  DP  is  clearly  necessary  and  definitely  the 
responsiblity  of  both  user  and  DP  management.  In  the  abstract  a  strategic 
planning  process  is  easy  to  specify.  In  the  specific,  most  firms  do  not 
follow  this  process  because  it  is  essentially  content-free  and  dependent 
upon  the  specificity  of  corporate  strategic  plans  and  their  indeterminate 
linkage  to  DP  activities. 

The  process  for  defining  DP  success  used  in  this  research  suggests  a 
practical,  concrete,  understandable  method  to  perform  the  necessary  first 
steps  in  strategic  planning  for  DP.  Based  upon  the  results  of  this  sample, 
it  is  undoubtedly  time  to  define  DP  success  at  the  firm  level,  diagnose 
causes,  and  assign  priorities  for  improvement. 


*This  is  truer  for  the  six  collectively  than  for  some  firms  in  particular. 
The  lower  the  users'  ratings  of  success,  the  greater  the  need  for 
improving  co-joint  problem  definition. 
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Summary  of  Results 

There  is  a  problem  in  DP  success,  it  is  widespread  and  severe. 
DP  and  user  managers  have  excellent  agreement  on  the  relative 
success  of  the   26  criteria  studied. 

The  pattern  of  relative  strengths  and  weaknesses  is  the  same  for 
all  firms.  Moreover,  there  is  excellent  agreement  between  DP  and 
users  on   this   pattern. 

Relative  strength  exists  in  technical,  operational,  efficiency- 
oriented  criteria.  Development  of  new  managerial  systems,  user 
relations,  and  the  managerial  effectiveness-oriented  criteria  are 
comparatively  weak. 

The  heart  of  the  problem  lies  not  in  importance  or  performance 
disagreements  between  users  and  DP,  rather  in  the  more  basic 
imbalance  between  importance  and  performance.  Both  DP  and  users 
fail  to  detect  any  desirable  or  justifiable  relationship  between 
importance  and  performance  on  the  criteria   for   success. 

There  are  too  many  areas  of  desired  improvement  for  the  scarce 
resources  available,  especially  management  attention,  to 
sufficiently  address  them  all  simultaneously.  Priorities  for 
action  can  be  determined  by  the  basic  cause  of  low  success  — 
gaps  between  importance  and  performance.  Large  importance- 
performance  gaps  on  ten  criteria  have  been  identified  with 
excellent   agreement  between  DP  and  user  management. 

Summary  of  Recommendations 

The  process  chosen  for  DP  strategic  planning  is  critical.  It 
must  strongly  link  corporate  needs  with  DP  functions  or  criteria, 
provide  an  integrated  definition  of  DP  success,  and  be  practical 
enough  for   annual   planning. 

Based  upon  an  integrated  definition  and  measurement  of  success, 
areas  of  needed  improvement  should  be  identified,  agreed  upon, 
and      prioritized.  Resource      availability      and       co-joint 

responsibilities   in   improvements  should  be  carefully  considered. 

Objectives  must  be  set  for  each  prioritized  criteria  and  the 
overall  improvement  effort.  Plans  should  include  reaching  out 
for  fresh  inputs,  possible  precedence  relationships  between 
criteria,  available  resources,  impacts  on  involved  personnel,  and 
the  organization's  capacity  for   change. 

The  progress  of  improvement  efforts  should  be  monitored, 
resources  reallocated  as  necessary,  and  a  practical  process  for 
annual   re-definition  and  measurement  of  success  instituted. 


DP  is  both  too  expensive  and  too  critical  for  mis-allocated 
improvement  efforts.  Defining  success  for  DP  is  a  necessary  first  step  in 
strategic   planning   for  effective  improvement  of  DP. 
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Exhibit  1 
26  Criteria  of  DP  Success 

Success   Group  and  Criteria 

Technical 
4.7     3.  developing  more  monitor  systems 

4.6  F.  quality  of  DP  systems  analysts 

5.0  H.  technical  competence  of  the  DP  staff 
4.8 

Security 

4.7  E.  data  security  and  privacy 

Operations 
4.4     X.  running  current  systems  (costs,  ease  of  use,  maintenance) 

4.4  W.  availability  and  timeliness  of  report  delivery  to  users 
4.7    B.  efficiency  of  hardware  utilization 

4.3     C.  hardware  and  systems  downtime 

4.6  Z.  DP  profitability  from  user  billbacks  and  billable  time  ratios 
4.4 

Proportion 

4.5  R.  sophistication  of  new  systems 

4.1  Q.  increasing  the  proportion  of  DP  effort  expended  in  creating  new  systems 
4.3 

Cost /Direction 

4.2  P.      overall   cost-effectiveness 

3.7  N.      involvement  of  senior  users  in  DP  policy  formulation  and   evaluation. 
3.9 

User  Relations 

3.6  T.  user  oriented  systems  analysts  who  know  user  operations 
4.0     V.  appropriate  DP  budget  size  or  growth  rate 

4.0    A.  communication  with  managerial  uses 

3.8  0.  responsiveness  to  user  needs 

3.8     U.  DP  support  for  uses  in  preparing  proposals  for  new  systems 
3.8 

Development 

4.2  Y.  report  contents  (relevance,  currentness,  flexibility,  accuracy) 

2.7  D.   Training  Programs  for  users  in  DP  capabilities 

3.8  G.  attitudes  of  users  to  DP 

4.0     K.  developing  more  exception  systems 

3.3  L.   developing  more  inquiry  systems 

3 .2  M.  developing  more  analysis  systems 
3.5 

Backlog/Process 

3.3  I.      the  new  system  request  backlog 

3.7  S.      improving   new  system  development   (time,   cost,   quality,   disruptions) 

3.5 

*  Average  success  rating   for  all   user  managers. 
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Exhibit  2 
Profile  of  Manufacturing  Firms  Surveyed 


Firm 

Industry 

Sales* 

DP 

Budget* 

(oon,ooo) 

(000,000) 

1 

equip  Mfg 

110 

2.2 

2 

analog   elec 

50 

1.0 

3 

diverse  mfg 

240 

2.4 

4 

chemical 

65 

0.5 

5 

aerospace 

700 

5.5 

6 

digital   elec 

67 

0.36 

DPS  Sales* 

Respondents 

2% 

11 

2% 

34 

1% 

16 

0.8  % 

16 

0.8  % 

12 

0.53% 

25 

114 

*not  parent  corporation,  within  boundary  of  user  community  or  division  studied 


Exhibit  3 
Profile  of  Survey  Respondents 

Level*       DP        Mfg         Fin      Total  Average  Salary 

($000) 

5           4         15  44 

11          12         25  31 

21           9         39  25 

13           6         34  20 


1 

2 
3 
4 

6 

3 

9 

15 

total 

33 

response 
rate 

97!K 

50  31        114 

91%  89%        92? 

*hierarchical  level  within  user  community 


Exhibit  4 
Ratings  of  Overall  Success  in  DP 


very 

poor  inadequate  good  excellent 


Firms  L  2.  2.  ^  ^  L  Average 

DP  Managers  5.1  4.3  4.0  4.0  4.4  4.6  4.4 
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EXHIBn  5 

Comparison  of  Success  by  Group  of  Criteria 

Firm  5  and  Firm  3 


7  -I 


6  _ 


5  _ 


4  - 


2  « 


7  excellent 


5  good 


3  inadequate 


1  very  poor 


Firm  5  (white  bars)  users'  rating  of  DP  success  =  4.8 
Firm  3  (hatched  bars)  users'  rating  of  DP  success  =  3.2 


EXHIBIT  6 

COMPARISON  OF  DP  AND  USERS  RATINGS  OF  SUCCESS 
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USER  MANAGERS'  RATINGS  OF  DP  SUCCESS 


EXHIBIT  7 
COMPARISON  OF  DP  and  USERS  RATINGS  OF  IMPORTANCE 
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USER  MANAGERS'  RATINGS  OF  IMPORTANCE 


EXHIBIT  8 
COMPARISON  OF  USERS'  RATINGS  OF  IMPORTANCE  AND  PERFORMANCE 
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EXHIBIT  9 
COMPARISON  OF  DP^s  RATINGS  OF  IMPORTANCE  AND  PERFORMANCE 
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EXHIBIT  10 
COMPARISON  OF  FIRM  5  USERS'  IMPORTANCE  AND  PERFORMANCE 
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FIRM  5  users'  IMPORTANCE  RATINGS 


EXHIBIT  II 
COMPARISON  OF  FIRM  5  USERS-  IMPORTANCF  AND  PERFORMANCE 
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FIRM  3  users'  importance  RATINGS 
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EXHIBIT  12 
Importance  -  Performance  Gaps  for  DP  and  Users 
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